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PREFACE 


This  document  was  prepared  under  Central  Research  project  (CRP)  1072.  It 
summarizes  the  results  of  a  simulated  MOUT  experiment  in  which  situational  awareness 
was  varied  in  a  controlled  manner.  Using  the  Soldier  Visualization  Station  (SVS)  and 
Dismounted  Infantry  Semi -Automated  Force  (DISAF)  simulations,  this  experiment  was 
conducted  in  the  IDA  Simulation  Center  during  the  spring  and  summer  of  2000. 

The  authors  would  like  to  thank  Dr.  John  M.  Gray  and  Dr.  Robert  F.  Richbourg 
for  their  reviews  of  this  document.  Their  insight  provided  much-needed  guidance.  The 
authors  would  also  like  to  thank  Mr.  Stanley  A.  Weber  for  his  technical  assistance  in 
maintaining  the  simulation  equipment. 
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EXECUTIVE  SUMMARY 


A.  BACKGROUND 

In  the  past  decade,  the  United  States  has  taken  on  a  number  of  peacekeeping  roles 
and  missions.  As  this  trend  is  likely  to  continue  for  the  foreseeable  future,  Military 
Operations  in  Urban  Terrain  (MOUT)  have  become  an  area  of  intense  study  and  analysis. 
MOUT  sites  for  training  are  planned  or  have  been  constructed  at  several  military 
installations.  These  include  Fort  Benning,  Camp  Lejeune,  29-Palms,  Fort  Drum, 
Quantico,  Fort  Lewis,  Fort  Polk,  Fort  Campbell,  and  Fort  Knox. 

A  central  MOUT  issue  is  situational  awareness  (SA),  or  the  understanding  of  the 
military  condition  at  hand.  It  includes  the  ability  to  quickly  and  correctly  locate  and 
identify  hostile  forces,  own  forces,  and  neutral  non-combatants.  As  urban  operations 
often  take  place  within  buildings  that  may  contain  any  or  all  of  these  categories  of 
personnel,  MOUT  presents  particularly  difficult  situational  awareness  issues. 

B.  PURPOSE 

This  paper  is  an  investigation  into  the  utility  of  enhanced  situational  awareness  in 
military  operations  in  urban  terrain.  It  is  a  continuation  of  two  previous  studies  that 
examined  the  impact  of  various  levels  of  infonnation  for  the  warfighter  in  small  unit 
operations  [Refs.  1,  2],  It  is  based  upon  a  series  of  simulation  trials  in  which  two 
fireteams  of  dismounted  infantry  (DI)  attempt  to  wrest  control  of  a  building  on  the 
McKenna  MOUT  site  from  an  entrenched  hostile  force.  This  investigation  was 
undertaken  in  order  to  better  understand  the  way  in  which  enhanced  situational  awareness 
could  save  lives  and  reduce  casualties  under  conditions  in  which  a  relatively  small  Blue 
unit  is  required  to  enter  and  secure  buildings  in  counter-insurgency  operations. 

A  second  purpose  of  this  paper  is  to  evaluate  the  Soldier  Visualization  System 
(SVS)  and  Dismounted  Infantry  Semi-Automated  Forces  (DISAF)  as  tools  for  MOUT- 
related  analyses.  Indeed,  the  underlying  experiment  was  a  vehicle  for  familiarizing  the 
study  team  with  these  simulations.  Briefly,  both  are  interactive  distributed  simulations. 
DISAF  enables  users  to  create  path  networks  along  which  infantry  fireteams  can  walk, 
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crouch,  or  crawl,  both  indoors  and  outdoors.  While  moving  along  these  networks,  the 
members  of  these  fireteams  can  engage  members  of  hostile  units  (other  dismounted 
infantry  in  this  case),  change  posture,  or  alter  their  routes  and  seek  protective  cover. 
Engagement  behavior  is  automatic,  whereas  the  user  makes  posture  and  route  selections. 
DISAF  provides  a  two-dimensional  “top”  view  (commonly  known  as  “third  person”)  of 
the  terrain,  buildings,  and  entities. 

SVS  simulates  the  detailed  kinematics  of  individual  dismounted  infantrymen. 
Each  SVS  entity  is  controlled  by  a  player  who  uses  a  joy-stick  to  impart  translational 
motion,  alter  posture,  and  fire  weaponry.  Unlike  DISAF,  SVS  infantrymen  do  not 

automatically  engage  opponents - but  are  dependent  on  the  user  to  aim  and  fire 

simulated  small  arms  associated  with  the  DI.  SVS  provides  the  user  with  a  “stealth  view” 
(or  “first  person”  in  modern  parlance)  of  the  MOUT  site  and  its  combatants. 

C.  TEST  PLAN 

This  experiment  consisted  of  a  series  of  20  McKenna  MOUT  site-based  trials  in 
which  half  were  conducted  with  an  enhanced  level  of  situational  awareness  available  to 
the  Blue  team  and  half  were  conducted  at  nominal  levels.  Enhanced  situational 
awareness  meant,  for  purposes  of  this  test,  that  the  Blue  team  had  perfect  knowledge  of 
its  own  and  the  Red  force  locations.  Nominal  situational  awareness  meant  that  the  Blue 
team  knew  only  its  own  force  location  and  the  position  of  Red  entities  detected  by  the 
Blue  force. 

The  Blue  team  operated  DISAF  and  the  Red  team  operated  SVS.  In  a  given  trial, 
the  Blue  team  was  made  up  of  two  members  of  the  study  team,  each  controlling  one  four- 
man  fireteam,  while  the  Red  team  consisted  of  three  study  team  members  (each 
controlling  one  SVS  infantryman).  All  combinations  of  two  and  three  member  teams 
from  the  five  available  operators  played  at  high  and  low  SA  levels  during  the  course  of 
the  experiment. 

The  Red  team  occupied  the  first  floor1  of  Building  A  on  the  McKenna  MOUT 
site.  They  assumed  defensive  positions  wherever  they  saw  fit.  The  Blue  force 
approached  Building  A  from  behind  the  outlying  McKenna  buildings  or  its  various  tree 


1  The  restriction  to  the  first  floor  was  somewhat  unsatisfying  as  the  rooftop  is  often  a  preferred  defensive 
position.  This  restriction  was  enacted  largely  to  accommodate  certain  limitations  of  the  models. 
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lines.  The  Blue  force  attempted  to  enter  the  building  and  dislodge  Red.  The  game  ended 
when  all  the  members  of  one  team  were  disabled. 

D.  OUTCOMES 

The  Blue  team  suffered  18  more  casualties,  or  approximately  2  more  per  game  on 
average,  under  low  conditions  than  under  enhanced  situational  awareness  (Blue  lost  a 
total  of  55  personnel  in  the  course  of  10  low  SA  games  and  37  in  10  high  SA  games).  Of 
these,  14  casualties  occurred  while  approaching  the  building  and  four  occurred  inside. 
Red  lost  at  least  two  combatants  in  every  game.  Blue  losses  varied  between  zero  and 
eight.  Figure  ES-1,  in  game  order  by  Blue  team,  summarizes  the  trial  outcomes. 


Figure  ES-1.  Blue  Casualties 

Figure  ES-1  shows  that  Blue  teams  suffered  fewer  casualties  in  four  games  in 
which  they  had  enhanced  situational  awareness  (labeled  “High  SA”  in  the  chart),  than  in 
the  corresponding  low  conditions  games,  suffered  more  in  three  games,  and  the  same 
number  three  times  (Games  4,  7,  and  9).  A  non-parametric  pairwise  comparison 
(Wilcoxon  Signed  Ranks  Test)  of  these  data  indicates  that  the  difference  in  Blue 
casualties  under  enhanced  and  low  SA  conditions  is  not  significant. 

Table  ES-1  displays  the  number  of  wins  and  losses  over  the  course  of  the  20 
experimental  trials.  Blue  was  victorious  in  8  of  the  10  games  in  which  it  enjoyed 
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enhanced  situational  awareness,  but  split  the  low  games  evenly  with  Red.  No  games 
ended  in  ties. 


Table  ES-1.  Wins  and  Losses 


Blue  Team 

Test 

Condition 

Total 

Low  SA 

High  SA 

Win 

5 

8 

13 

Loss 

5 

2 

7 

Total 

10 

10 

20 

While  these  data  appear  to  indicate  that  SA  has  a  strong  effect,  they  are 
insufficient  to  reject  the  null  hypothesis  that  wins  and  losses  are  independent  of  test 
condition  (the  Fisher-Irwin  Exact  Test  is  not  significant).  Indeed,  under  the  null 
hypothesis,  the  expected  number  of  Blue  wins  in  “High  SA”  is  6.5  and  the  expected 
number  of  losses  is  3.5.  A  larger  set  of  trials  may  have  resulted  in  more  definitive 
outcomes. 

E.  TACTICAL  ANALYSIS 

We  conducted  an  examination  of  Blue  tactics  to  determine  the  reason  Blue  odds 
of  surviving  increased  with  enhanced  situational  awareness.  Each  trial  was  reviewed 
from  datalogger  files  and  we  analyzed  the  behavior  of  Blue  forces  with  respect  to  unit 
coherence  and  coordination  with  which  missions  were  executed.  In  the  course  of 
reviewing  these  files,  a  striking  difference  between  the  enhanced  and  nonnal  games 
became  apparent.  During  the  low  SA  games,  Blue  tended  to  make  use  of  “scouts,”  small 
one  or  two-man  patrols  that  would  approach  Building  A  with  the  intent  of  locating  the 
Red  combatants.  These  scouts  tended  to  become  casualties.  Indeed,  18  Blue  casualties 
were  associated  with  this  scouting  mission  (14  while  approaching  the  building,  4  while 
inside). 

On  the  other  hand,  when  Blue  enjoyed  full  visibility  (that  is,  enhanced  situational 
awareness),  it  tended  to  reach  the  building  safely.  Under  these  conditions,  Blue  attacked 
the  various  rooms  in  a  synchronized  manner,  often  engaging  a  target  from  two  directions 
simultaneously — or  nearly  so.  When  it  was  not  possible  to  execute  this  pincer  maneuver, 
Blue  would  attempt  to  “rush”  Red  combatants  with  several  fireteam  members.  In  low  SA 
games,  once  it  learned  the  Red  locations,  Blue’s  tactics  became  much  the  same  as  in  the 
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enhanced  SA  trials.  It  exercised  its  missions  with  the  same  unit  cohesion  and  suffered 
casualties  at  no  greater  rate  than  under  the  more  favorable  conditions. 

F.  CONCLUSIONS 

1.  Models 

a.  DISAF 

The  operator’s  ability  to  exercise  fine  control  and  to  react  to  changes  was  found  to 
be  very  limited  using  DISAF.  In  previous  studies  [Refs.  1,  2]  of  tank  battles  using 
Modular  Semi-Automated  Forces  (ModSAF),  a  “MOVE”  command  (to  a  specific  point 
on  the  map  display,  selected  by  a  mouse  click)  was  commonly  used  to  quickly  direct 
tanks  and  override  any  previous  commands.  This  command  did  not  function  properly  for 
individual  soldiers  (or,  at  least,  the  study  team  was  unable  to  make  it  work).  In  these 
experiments,  operators  constructed  paths  and  submitted  combat  instructions  for  the 
soldiers  to  follow  these  paths.  This  process  was  cumbersome  and  commands  were  not 
easily  changed  or  overridden. 

DISAF  has  a  few  complex  behaviors,  but  these  were  not  found  to  be  useful.  For 
example,  there  is  a  “CLEAR  ROOM”  command  that  was  not  used.  More  generally,  each 
operator  elected  not  to  use  commands  to  the  entire  fire  team2  and  instead  elected  to 
control  each  member  individually.  This  was  more  cumbersome  and  time  consuming  but 
was  the  only  way  to  achieve  coordination  adapted  to  the  situation.  Even  so,  coordination 
was  difficult.  Other  than  the  room  clearing  order,  there  were  no  commands  that  directed 
soldiers  to  act  together,  e.g.,  to  move  or  to  attack  when  another  soldier  reaches  a  certain 
position. 

Some  tactics  that  would  have  been  logical  to  try  to  use  were  not  because  DISAF 
did  not  have  the  tools.  Perhaps  the  best  example  of  this  is  the  inability  to  use  protected  or 
semi-protected  positions.  It  was  not  possible  to  quickly  peer  around  corners  or  to  find  a 
partly  protected  location  to  fire  from  (this  severely  limits  the  model’s  realism). 

In  summary,  DISAF  does  not  seem  well  adapted  to  the  rapid,  precise,  and 
dynamic  movements  and  decisions  that  might  be  required  in  a  MOUT  environment.  For 


2  The  number  of  combatants  controlled  differed  depending  on  whether  one  operated  DISAF  or  SVS. 
While  this  may  have  introduced  a  disparity  in  workload,  it  did  not  affect  the  measurement  of  the  impact 
of  the  test  variable  situational  awareness. 
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these  experiments,  the  DISAF  operators  constructed  planned  maneuvers,  which  worked 
reasonably  well  when  enemy  locations  were  known  and  the  enemy  soldiers  were  not 
moving. 

b.  SVS 

This  experiment  revealed  significant  SVS  limitations.  Probably  the  greatest  is 
“awareness.”  The  single  computer  screen  does  show  much  peripheral  vision,  and  it  is  not 
possible  to  move  the  soldier’s  head  (separately  from  his  body)  to  look  around.  The 
controls  are  somewhat  “jumpy,”  so  that  controlling  fine  motion  and  aiming  at  small 
targets  can  be  difficult. 

In  addition,  simulation  implementation  problems  exist.  Soldiers  can  become 
“stuck”  in  walls  if  they  move  too  close  to  them,  and  may  even  protrude  through  the  other 
side  (where  they  are  vulnerable).  During  these  exercises,  SVS  would  freeze  occasionally 
for  a  few  seconds,  apparently  when  network  traffic  was  heavy. 

In  summary,  SVS  is  much  better  than  DISAF  in  responding  to  individual  control 
but  nonetheless  has  limitations  that  affect  the  user’s  ability  to  conduct  (moderately 
realistic)  simulated  MOUT  operations. 

2.  Situational  Awareness 

The  limitations  of  the  simulators  and  the  use  of  a  single  limited  scenario  precludes 
drawing  general  conclusions  about  how  to  conduct  MOUT  operations  or  the  absolute 
value  of  awareness  of  enemy  positions.  Instead,  this  small  experiment  suggests  ideas  that 
might  be  analyzed  more  thoroughly,  including: 

•  Tactics  might  vary  depending  on  situational  awareness,  and  more  effective 
tactics  might  be  used  with  greater  situational  awareness. 

•  The  unprotected  approach  to  a  defended  building  is  highly  dangerous  and 
might  benefit  greatly  from  awareness  of  enemy  locations  within  the  building. 
(It  should  be  noted  that  in  these  experiments  the  defending  force  did  not  have 
sufficient  manpower  to  cover  all  possible  approach  paths.) 

In  these  experiments,  the  Blue  force  used  much  better  coordinated  attacks  when 
the  Red  positions  were  known  than  when  they  were  not.  When  the  positions  were  not 
known,  Blue  actions  tended  to  be  exploratory  and  frequently  resulted  in  one  or  more 
scout  soldiers  getting  killed.  When  positions  were  known,  Blue  seldom  used  only  one 
soldier  to  attack  an  enemy  position. 
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The  difference  in  tactics  was  most  notable  in  approaching  the  building.  If 
positions  were  known,  Blue  would  generally  approach  the  building  from  an  unobserved 
direction  and  mount  a  coordinated  attack  after  reaching  the  building  perimeter.  If  enemy 
positions  were  not  known,  Blue  would  most  often  send  soldiers  individually  to  approach 
the  building,  until  a  safe  approach  path  was  found. 

These  experiments  did  not  resolve  questions  involving  tactics  and  effectiveness 
within  the  building.  In  particular,  Blue  losses  in  approaching  the  building  without 
situational  awareness  might  have  occurred  within  the  building  (in  discovering  enemy 
positions),  even  if  the  force  was  able  to  approach  without  losses.  Thus  no  accurate  and 
unbiased  measurement  was  taken  of  Blue’s  vulnerability  without  situational  awareness 
within  the  building. 
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I.  BACKGROUND  AND  INTRODUCTION 


A.  PURPOSE 

This  paper  is  an  investigation  into  the  utility  of  enhanced  situational  awareness  in 
military  operations  in  urban  terrain.  It  is  based  upon  a  series  of  simulation  runs  in  which 
two  firctcams  of  dismounted  infantry  (DI)  attempt  to  wrest  control  of  a  building  on  the 
McKenna  MOUT  site  from  an  entrenched  hostile  force.  This  investigation  was 
undertaken  to  better  understand  the  way  in  which  enhanced  situational  awareness  could 
save  lives  and  reduce  casualties  under  conditions  in  which  a  relatively  small  Blue  unit  is 
required  to  enter  and  secure  buildings  in  counter-insurgency  operations. 

A  secondary  purpose  of  this  paper  is  to  evaluate  the  Soldier  Visualization  System 
(SVS)  and  Dismounted  Infantry  Semi-Automated  Forces  (DISAF)  as  tools  for  MOUT- 
related  analyses. 

B.  MCKENNA  MOUT  SITE 

The  McKenna  Military  Operations  in  Urban  Terrain  (MOUT)  site  in  Fort  Benning 
Georgia,  is  a  practice  ground  for  Military  Operations  in  Built-up  Areas  (MOBA).  As  a 
result  of  a  Defense  Science  Board  (DSB)  study  in  1994,  the  Army's  Topographic  and 
Engineering  Center  (TEC)  in  Fort  Belvoir,  Virginia,  created  a  digital  terrain  database  of 
the  MOUT  site  that  could  be  used  in  simulations.  This  database  uses  a  higher  resolution 
than  is  common  for  these  sorts  of  databases.  The  results  of  the  MOBA  Terrain  Database 
project  are  posted  on  the  web  at  http://www.tmpo.nima.mil/news/moba.html. 

We  selected  this  terrain  for  the  experiment  for  several  rasons:  (1)  it  was  the  only 
synthetic  terrain  available  over  a  real  practice  MOUT  site  in  the  IDA  Simulation  Center; 
(2)  it  had  sufficient  number  of  and  detail  in  and  buildings  to  create  interesting  and 
insightful  scenarios;  (3)  versions  of  this  terrain,  compatible  with  both  the  SVS  and 
Dismounted  Infantry  Semi -Automated  Forces  (ModSAF),  were  available. 
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C.  SVS  AND  DISAF 

As  suggested  in  the  previously,  we  performed  this  study  using  two  interactive 
simulations,  SVS  and  DISAF,  which  we  selected  largely  on  the  basis  of  supporting  a 
representation  of  the  McKenna  MOUT  site’s  terrain.  More  specifically,  each  simulation 
allows  the  user  to  conduct  free  play  engagements  in  and  around  certain  of  the  McKenna 
buildings. 

DISAF  enables  users  to  create  path  networks  along  which  infantry  fireteams  can 
walk,  crouch,  or  crawl  both  indoors  and  outdoors.  While  moving  along  these  networks, 
fireteam  members  can  engage  hostile  units  (other  dismounted  infantry  in  this  case), 
change  posture,  or  alter  their  routes  and  seek  protective  cover.  Engagement  behavior  is 
automatic,  whereas  the  user  chooses  posture  and  routes. 

DISAF  provides  a  two-dimensional  view  of  the  MOUT  site,  showing  the 
buildings’  interior  rooms  and,  depending  on  test  conditions,  locations  of  combatants.  A 
sample  image  of  the  site,  showing  the  interior  of  Building  10,  appears  in  Figure  1. 

SVS  simulates  the  detailed  kinematics  of  individual  dismounted  infantrymen. 
Each  SVS  entity  is  controlled  by  a  player  who  uses  a  joy-stick  to  impart  translational 
motion,  alter  posture,  and  fire  weaponry.  Unlike  DISAF,  SVS  infantrymen  do  not 
automatically  engage  opponents  but  are  dependent  on  the  user  to  aim  and  fire  simulated 
small  arms  associated  with  the  DI.  SVS  provides  the  user  with  a  “stealth  view”  of  the 
MOUT  site  and  its  combatants.  That  is,  unlike  the  two-dimensional  DISAF  view,  the 
SVS  user  can  see  the  battleground  from  the  perspective  of  the  entity  being  controlled. 
This  is  a  much  more  local  but  ever-changing  view  of  buildings,  vegetation,  and  people. 

The  analysis  team  was  familiar  with  DISAF  from  work  on  previous  studies. 
While  less  than  enthusiastic  about  DISAF  capabilities,  the  team  believed  this  simulation 
could  provide  the  basic  functions  necessary  to  address  the  core  issues  in  this  MOUT 
analysis.  On  the  other  hand,  the  analysis  team  wanted  to  find  new  models  with  which  to 
conduct  studies  of  this  type.  Because  SVS  was  available  on  a  number  of  Simulation 
Center  workstations,  it  became  a  candidate  simulation  for  this  task.  After  a  certain 
amount  of  “pilot  testing,”  the  analysis  team  detennined  that  SVS  and  DISAF  were 
compatible  in  the  sense  that  they  interacted  in  a  tractable  manner  and  yielded  what 
appeared  to  be  credible  results  when  used  together  in  simple  MOUT  scenarios. 


2 


UNCLASSIFIED 


UNCLASSIFIED 


Figure  1.  Building  Interior  at  the  McKenna  MOUT  Site 


D.  SITUATIONAL  AWARENESS:  VISIBILITY  AS  PROXY 

The  primary  purpose  of  this  task  was  to  measure  the  effect  of  enhanced  situational 
awareness  on  urban  operations.  To  perform  these  measurements,  it  was  necessary  to 
define — in  terms  of  the  simulations  at  hand — the  notion  of  “enhanced  situational 
awareness.”  The  simplest  candidate  notion  involved  the  condition  in  which  all 
combatants  were  continuously  aware  of  the  locations  of  all  other  combatants  in  the 
engagement  at  hand. 

This  definition  was  particularly  appealing  because  it  was  both  an  upper  limit  on 
what  one  could  expect  to  know  in  a  battlefield  engagement,  and  it  was  reasonably  simple 
to  implement  using  the  DISAF  simulation.  Also,  it  provided  a  reasonable  alternative 
against  which  it  could  be  compared.  More  precisely,  DISAF  provides  a  menu-driven 
toggle  that  allows  players  to  switch  between  two  extreme  viewing  capabilities.  The  first 
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allows  players  to  see  all  combatants,  but  the  second  pennits  players  to  see  only  own-side 
entities  (and  those  within  line-of-sight). 

Because  no  similar  notion  applies  to  SVS  (which  provides  a  line-of-sight 
capability  but  has  no  two-dimensional  top  view  like  DISAF),  we  decided  to  use  DISAF 
as  the  Blue  counter-insurgency  team.  We  conducted  two  sets  of  trials:  one  in  which 
Blue  had  the  ability  to  see  all  combatants,  and  one  in  which  Blue  could  only  see  its  own 
side. 
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II.  GAME  SETUP  AND  EXECUTION 

A.  FACILITY  AND  TEAMS 

All  trials  were  conducted  in  the  IDA  Simulation  Center.  This  facility  houses  three 
SVS  workstations  and  more  than  a  half  dozen  Silicon  Graphics  workstations  (DISAF 
hosts).  Each  SVS  workstation  requires  one  operator/player  and  simulates  one 
dismounted  infantryman.  The  DISAF  workstations  also  require  one  operator  but  are 
capable  of  simulating  large  numbers  of  four-man  il relearns  or  nine-man  infantry  squads. 
Since  five  players  were  on  the  analysis  team,  the  availability  of  equipment  determined  the 
composition  of  the  insurgency  (Red  SVS)  and  counter-insurgency  (Blue  DISAF)  forces. 
The  SVS  insurgency  team  would  consist  of  three  combatants  (controlled  by  three 
operators)  and  DISAF  counter-insurgency  team  would  be  made  up  of  two  four-man 
II relearns  (controlled  by  two  operators). 

To  conduct  a  balanced  experiment,  all  10  possible  combinations  of  three  versus 
two-player  squads  were  formed  from  the  five  available  players.  Players  were  designated 
by  the  letters  “a”  through  “e,”  and  teams  were  designated  by  concatenating  the  letters  of 
their  members. 

B.  TRIAL  MATRIX 

A  total  of  20  trials  were  conducted:  10  with  enhanced  situational  awareness  (or 
“high  visibility”  as  described  in  Chapter  I,  Section  C)  on  the  Blue/DISAF  side  and  10 
with  standard  situational  awareness.  Recall,  under  standard  situational  awareness  (“low 
visibility”),  each  player  sees  only  the  members  of  his  own  team  and  the  opponents  within 
line-of-sight  of  entities  under  his  control.  Twenty  trials  permitted  each  team  to  play  once 
under  each  SA  condition. 

Table  1  summarizes  the  schedule  of  games.  The  10  groups  of  players  were  placed 
in  random  order.  Within  this  ordering,  each  team  played  two  games:  one  at  each  of  the 
SA  levels.  The  Blue  DISAF  team  selected  the  SA  they  wanted  to  play  first  and  withheld 
this  choice  from  their  SVS  opponents  until  both  games  were  completed.  Of  course,  the 
subsequent  game  was  conducted  at  the  opposite  level.  The  “within- team”  ordering  is 
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shown  in  the  third  column  where  the  first  of  a  pair  is  shown  at  the  left  boundary  of  the 
column  and  the  second  is  shown  on  the  right. 


Table  1.  Trial  Matrix 


Trial 

Date 

Blue-Red  Teams 

SA  Level 

1 

5/18/00 

CD -ABE 

high 

2 

5/18/00 

CD -ABE 

low 

3 

5/18/00 

AB - CDE 

low 

4 

5/18/00 

AB - CDE 

high 

5 

5/26/00 

BD  -  ACE 

high 

6 

5/26/00 

BD  -  ACE 

low 

7 

5/26/00 

AE -  BCD 

high 

8 

5/26/00 

AE -  BCD 

low 

9 

6/14/00 

BC  -  ADE 

low 

10 

6/14/00 

BC  -  ADE 

high 

11 

6/19/00 

CE  -  ABD 

low 

12 

6/19/00 

CE  -  ABD 

high 

13 

6/19/00 

AD  -  BCE 

high 

14 

6/19/00 

AD  -  BCE 

low 

15 

6/23/00 

DE  -  ABC 

high 

16 

6/23/00 

DE  -  ABC 

low 

17 

6/23/00 

AC  -  BDE 

low 

18 

6/23/00 

AC  -  BDE 

high 

19 

7/28/00 

BE  - ACD 

low 

20 

7/28/00 

BE  - ACD 

high 

Half  of  the  enhanced  SA  games  were  played  first  and  half  of  the  low  SA  games 
were  played  first.  This  was  fortuitous  in  that  it  provided  a  convenient  means  of  testing 
whether  or  not  order  (within  a  team’s  pair  of  games)  affected  outcomes.  This  balance 
was  not  pre-preordained,  however,  as  will  be  discussed  in  Chapter  II,  Section  C 

Typically  two  or  four  games  were  played  in  one  session  (day).  Each  game 
required  approximately  20  minutes  to  complete. 

C.  CONDUCT  OF  THE  GAMES 

The  next  few  paragraphs  are  a  short  description  of  the  manner  in  which  the  games 
were  played.  Ground  rules,  player  interaction  and  coordination,  and  a  number  of  other 
factors  that  dictated  game  conduct  are  related  in  detail.  This  section  is  intended  to 
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provide  the  reader  with  some  background  and  insight  so  the  remainder  of  the  paper  can 
be  read  in  the  proper  context. 

Each  game  was  a  contest  between  two  teams  of  players  controlling  entities 
representing  opposing  sides  in  a  localized  urban  combat  scenario.  All  games  were  played 
in  and  around  a  single  building  that  was  occupied  by  three  Red  SVS  infantrymen 
(controlled  by  three  operators).  Two  Blue  DISAF  four-man  fireteams  (controlled  by  two 
operators)  attempted  to  dislodge  them.  Games  were  played  until  all  members  of  one  team 
became  casualties. 

The  Red  and  Blue  operators  were  located  in  separate  rooms.  Communication 
between  the  rooms  (to  synchronize  the  start  of  trials,  etc.)  was  over  the  local  area  network 
or  the  phone.  In  this  way,  discussions  concerning  coordinated  tactics  and  test  condition 
would  not  be  heard  by  the  opposing  team. 

At  the  start  of  a  trial,  each  side  would  enter  its  area,  activate  its  simulation 
software,  and  discuss  battle  plans.  When  ready  to  begin,  each  side  would  signal  the  other 
(usually  by  phone).  When  both  teams  were  prepared,  the  Blue  team  would  begin  the 
“datalogger,”1  and  the  battle  would  begin.  From  this  point  on,  games  were  “free  play”  in 
the  sense  that  there  were  no  prescribed  scripts  to  follow  and  each  player  was  permitted  to 
control  his  combatants  as  he  saw  fit.  However,  some  ground  rules  were  applied  to  “level 
the  playing  field”  between  the  two  simulations;  others  pertained  to  the  information 
available  to  each  side  regarding  the  test  conditions.  These  rules  and  other  factors 
influencing  the  trials  are  discussed  in  the  next  few  sections. 

1.  General  Rules 

The  two  simulations,  SVS  and  DISAF,  differ  in  a  number  of  critical  respects. 
SVS  is  much  more  dependent  on  human  intervention,  while  DISAF  has  a  great  deal  of 
autonomous  behavior.  Not  all  characteristics  favored  DISAF  however.  For  example, 
SVS  entities  had  greater  capacity  for  ascending  stairs  (Building  A  has  two  stories),  and, 
in  certain  cases,  could  move  through  walls.  To  conduct  a  trial  in  a  reasonable  manner, 
certain  rules  were  imposed  to  compensate  for  model  differences.  Other  rules  were 
imposed,  not  because  of  model  advantages,  but  simply  to  limit  the  scope  and  duration  of 
the  engagements. 


1  The  “datalogger”  software  records  protocol  data  units  broadcast  by  the  simulations. 
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Ground  rules  prohibited  the  following  Red  (SVS)  activities: 

•  Entering  the  second  floor 

•  Leaving  the  building,  except  to  change  rooms 

•  Moving  through  walls. 

The  first  rule  was  imposed  because  of  a  limitation  in  DISAF  that  made  the 
transition  to  the  second  floor  difficult  to  execute.  Thus,  this  rule  eliminated  what  was 
perceived  as  an  unfair  advantage  for  Red.  The  second  rule  was  enforced  to  contain  the 
activity.  Only  one  building  on  the  McKenna  site  is  modeled  with  a  complete  interior 
(Building  A),  so  engagements  were  restricted  to  that  building  in  order  to  retain  the 
MOUT  character  of  the  experiment.  Also,  allowing  outdoor  activity  would  likely 
prolong  the  length  of  the  individual  trials.  The  third  rule  was  imposed  because  SVS  has 
the  ability  to  move  from  room  to  room  without  the  benefit  of  a  doorway,  whereas  DISAF 
does  not. 

The  following  rules  were  imposed  on  Blue  (DISAF): 

•  DI  perform  at  the  novice  level. 

•  Engagement  range  is  restricted  to  15  meters  or  less. 

•  Casualties  are  removed  from  the  “playing  field” 

Performance  was  restricted  to  the  novice  level  because  preliminary  testing 
indicated  DISAF ’s  normal  reaction  time  is  quicker  than  the  human  operator’s.  This  gave 
DISAF  an  apparent  advantage  in  otherwise  even  tactical  situations.  Range  limitation  was 
enforced  for  much  the  same  reason:  DISAF  is  capable  of  shooting  accurately  at  ranges 
well  over  a  kilometer,  while  the  SVS  operator  has  a  much  more  limited  capability. 
Casualties  were  removed  because  of  a  peculiar  DISAF/ModSAF  feature  that  allows 
entities  that  have  suffered  catastrophic  kills  to  “report”  the  location  of  enemy  units.  More 
precisely,  an  entity  is  displayed  on  a  Plan  View  Display  (PVD — the  graphical  interface 
for  the  workstation  hosting  DISAF)  when  an  object  belonging  to  the  local  force  (as 
defined  by  the  PVD)  is  within  line-of-sight  and  has  the  proper  orientation.  This  function 
is  not  curtailed  when  the  local  force  object  has  been  destroyed  or  killed.  To  prevent  Blue 
from  capitalizing  on  this  feature,  dismounted  infantry  were  removed  from  the  game  once 
they  were  killed. 
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2.  Secrecy  of  Visibility  Level 

In  addition  to  those  listed  above,  an  additional  rule  related  to  test  conditions  was 
imposed  on  the  players.  As  discussed  previously,  two  SA  conditions  were  examined  in 
these  trials.  Each  team  played  back-to-back  games,  one  at  high  SA  (visibility)  and  one  at 
low  SA  (although  the  order  in  which  teams  played  was  randomized).  The  Red  team  did 
not  know  which  condition  held  until  after  both  games  of  a  trial  pair  were  complete.  This 
information  was  withheld  because  it  was  believed  Red  might  change  tactics  and,  in  some 
manner,  confound  the  test  conditions.  (A  favorite  pastime  for  the  Red  SVS  players  was 
guessing  whether  Blue  had  visibility  during  the  first  or  second  trial  of  a  session.) 

3.  Technical  Difficulties 

One  aspect  of  simulation  trials  that  colors  the  manner  in  which  experiments  are 
conducted  is  the  ease  with  which  players  interact  with  the  software.  In  this  test,  technical 
difficulties  were  encountered,  but  all  were  manageable,  and  none  caused  any  sessions  to 
be  aborted  or  tests  to  be  rerun  (a  small  number  of  games  were  restarted,  however).  In 
particular,  the  DISAF  stations  would  sometimes  crash  and  their  entities  would  be  lost. 
When  this  happened,  the  operators  would  recover  as  necessary.  The  logger  would  not  be 
stopped  or  paused.  As  the  operators  gained  experience  and  proficiency  with  the 
equipment,  the  frequency  of  DISAF  crashes  diminished. 

On  several  occasions,  the  SVS  entities  became  stuck  in  the  walls  while  attempting 
to  navigate  through  the  rooms  of  Building  A.  This  is  a  shortcoming  of  SVS  that  allows 
DI  to  penetrate  walls  while  traversing  narrow  passageways  or  hunkering  in  stairwells. 
Sometimes  this  caused  the  DI  to  get  stuck  in  the  wall.  Occasionally  this  resulted  in  a  DI 
being  shot  because  he  was  partially  exposed  on  the  opposite  side  of  a  wall  he  intended  to 
hide  behind.  This  problem  could  generally  be  resolved  by  pressing  the  "go  through  wall" 
button  on  the  joystick. 

These — and  the  occasional  temporary  system  “freezes” — were  the  foibles  of  the 
game.  Beyond  these,  few  problems  occurred  and  the  sessions  generally  ran  smoothly  to 
completion. 
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III.  OUTCOMES  OF  THE  TRIALS 


A.  CASUALTIES 
1.  By  Visibility  Condition 

The  objective  of  the  simulation  trials  was  to  test  the  (null)  hypothesis  that 
enhanced  situational  awareness  has  no  effect  on  Blue  performance.  We  measured  Blue’s 
performance  by  the  number  of  casualties  suffered  and  the  number  of  missions 
successfully  completed  (we  defined  mission  success  as  defeating  all  members  of  the  Red 
insurgency  force).  Traditional  measures,  such  as  exchange  ratios,  were  shunned  because 
of  the  small  size  of  the  Red  force:  small  variations  in  Red  casualties  could  produce  large 
percentage  changes  in  the  ratio  of,  say,  Blue  to  Red  losses.  Also,  in  the  current  spirit  of 
peace-keeping  missions,  minimizing  Blue  casualties  is  likely  to  be  considered  more 
important  than  the  relative  rates  at  which  friendly  and  hostile  forces  are  diminished. 

Table  2  displays  the  number  of  Blue  casualties  and  survivors  for  the  two  test 
conditions.  Over  the  course  of  these  trials,  Blue  suffered  18  more  casualties  while  under 
low  visibility  than  under  the  enhanced  SA.  Table  2  indicates  that,  under  the  low 
condition,  the  probability  of  a  Blue  DI  surviving  the  MOUT  mission  was  only  about  32 
percent.  Under  the  higher  condition,  survival  probability  was  about  54  percent.  A  Chi- 
Squared1  test  applied  to  these  casualty  data  indicates  that  the  difference  in  the  odds  of 
survival  under  the  two  conditions  is  significant  below  the  1 -percent  level. 

It  is  interesting  to  note  however,  that  a  non-parametric  paired  comparison 
(Wilcoxon  Signed  Ranks  Test)  of  casualties  failed  to  show  a  significant  difference 
between  the  high  and  low  visibility  trials.  These  data,  paired  on  a  team-by-team  basis, 
are  displayed  in  Figure  2.  Although  this  may  be  paradoxical,  it  is  not  a  contradiction  of 
the  results  from  the  Chi-Squared  tests  because  the  former  concerns  the  relative  odds 
under  two  test  conditions,  and  the  latter  focuses  only  on  the  number  of  casualties. 


1  This  may  appear  to  be  a  cavalier  use  of  the  Chi-Squared  test  as  the  sampling  unit  was  the  trial  and  not 
the  dismounted  infantiy.  However,  the  perceived  difficulty  goes  away  because  all  trials  contained  the 
same  number  of  DI. 


11 


UNCLASSIFIED 


UNCLASSIFIED 


Table  2.  Blue  DISAF  Casualties  by  SA/Visibility  Condition 


Mortality 

Test  Condition 

Total 

Low  VIS 

High  VIS 

Dead 

55 

37 

92 

Live 

25 

43 

68 

Total 

80 

80 

160 

□  High  SA 
■  Low  SA 


123456789  10 


Game 


Figure  2.  Blue  DISAF  Casualties:  High  vs.  Low  Visibility 


Although  the  Blue  casualties  are  the  principal  focus,  the  number  of  Red  insurgent 
casualties  is  presented  for  completeness  in  Table  3.  Typically  these  varied  between  two 
and  three  for  each  game  and  showed  little  variation  over  the  course  of  the  trials. 
Statistical  tests  indicated  no  dependence  of  Red  outcomes  on  test  conditions. 


Table  3.  Red  SVS  Casualties  by  SA/Visibility  Condition 


Mortality 

Test  Condition 

Total 

Low  VIS 

High  VIS 

Dead 

25 

28 

53 

Live 

5 

2 

7 

Total 

30 

30 

60 
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2.  By  Game  Order 

Figure  2  shows  the  number  of  Blue  casualties  by  game  order.  While  no  obvious 
significance  appears  in  the  overall  order  of  play,  a  number  of  observations  can  be  made 
from  the  figure.  First,  three  of  the  Blue  teams  suffered  more  casualties  in  their  first  game 
than  in  their  second,  and  four  teams  suffered  more  in  the  second.  Three  teams  showed  no 
difference.  Also,  three  teams  did  worse  with  visibility  than  they  did  without  and  four 
teams  performed  better.  Three  teams  performed  equally.  Of  the  four  teams  showing 
improvement  with  visibility,  two  had  visibility  in  their  first  game  and  two  had  it  in  their 
second.  Of  the  three  that  did  worse,  two  had  visibility  in  their  second  game. 

The  probability  of  Blue  DI  surviving  the  first  game  were  somewhat  higher  in  the 
first  game  of  a  team  pair  than  in  the  second  (0.46  compared  to  0.39).  However,  Chi- 
Squared  tests  failed  to  indicate  any  significant  dependence  of  odds  of  survival  on  game 
order  within  pairs.  Indeed,  as  Table  4  demonstrates,  there  were  only  marginal  differences 
between  the  number  of  casualties  (or  survivors)  in  the  first  and  second  games. 
Unsurprisingly,  the  same  non-parametric  paired  comparison,  applied  above  with  respect 
to  visibility  conditions,  also  failed  to  reveal  any  distinction  based  on  order. 


Table  4.  Blue  DISAF  Casualties  by  Game  Order 


Mortality 

Game  Order 

Total 

First 

Second 

Dead 

43 

49 

92 

Live 

37 

31 

68 

Total 

80 

80 

160 

3,  By  Team 

For  completeness,  Red  team  outcomes  are  presented  by  order  within  team  pairs  in 
Table  5.  Outcomes  for  Red  show  no  significant  differences  in  Red  survival  odds  as  a 
result  of  order. 

A  non-parametric  test  (Friedman  Test  for  multiple  related  samples)  did  not  reveal 
any  significant  differences  among  the  10  Blue  teams.  However,  this  test  produced  a 
ranking  of  the  teams  based  on  the  number  of  casualties  in  two  trials,  which  appears  in 
Table  6  (lowest  ranking  is  best).  Also,  an  analysis  of  variance  (ANOVA)  showed  no 
strong  team  effects  or  interactions  among  team  players  in  these  trials. 
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Table  5.  Red  SVS  Casualties  by  Game  Order 


Mortality 

Game  Order 

Total 

First 

Second 

Dead 

26 

27 

53 

Live 

4 

3 

7 

Total 

30 

30 

60 

*  Not  significant 


Table  6.  Team  Rankings 


Team 

ae 

ab 

be 

bd 

ce 

ac 

be 

de 

cd 

ad 

Rank 

3.5 

3.75 

5 

5 

5 

5.25 

5.25 

6.25 

7.25 

8.75 

B.  WINS  AND  LOSSES 
1.  By  SA/Visibility  Condition 

For  purposes  of  these  trials,  a  successful  mission  from  the  Blue  perspective  was 
one  in  which  all  Red  combatants  were  defeated.  All  others  were  considered  victories  for 
the  insurgents.  Blue  won  the  overwhelming  majority  of  games  with  enhanced  SA. 
Indeed,  Blue  won  8  out  of  10  of  these  trials.  On  the  other  hand,  the  low  visibility  games 
were  split  equally,  with  five  won  by  Red  and  five  won  by  Blue  (see  Table  7).  Thus 
visibility  appears  to  be  a  strong  influence  on  outcome. 


Table  7.  Wins  and  Losses  by  SA/Visibility  Condition 


Mortality 

Test  Condition 

Total 

Low  VIS 

High  VIS 

Blue  Win 

5 

8 

13 

Blue  Loss 

5 

2 

7 

Total 

10 

10 

20 

From  a  technical  perspective,  however,  this  conclusion  is  optimistic.  For 
example,  the  two-sided  Fisher  Exact  Test  indicates  that  outcomes  at  least  this  extreme 
occur  in  about  one-third  of  the  cases.  While  this  may  appear  counter-intuitive,  it  is 
important  to  keep  in  mind  that  what  is  being  examined  here  is  not  whether  the  probability 
of  winning  is  0.50,  but  rather,  whether  or  not  visibility  makes  a  difference.  One  helpful 
insight  into  this  issue  is  to  observe  that  the  maximum  likelihood  estimate  of  probability  of 
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Blue  loss  is  0.35  under  the  (null)  hypothesis  that  no  difference  exists  between  test 
conditions.  With  this  probability,  outcomes  as  extreme2  as  those  in  Table  7  occur  about 
13  percent  of  the  time. 

2.  By  Game  Order 

Similarly,  the  order  in  which  games  were  played  did  not  have  a  great  impact  on 
the  odds  of  winning.  Table  8,  which  gives  the  total  wins  and  losses  by  order  within  team 
pair,  shows  that  relative  odds  vary  between  1.5  in  the  first  game  to  2.3  in  the  second. 
However,  due  to  small  sample  sizes,  this  difference  is  not  significant. 


Table  8.  Blue  Wins  and  Losses  by  Game  Order 


Outcome 

Game  Order 

Total 

First 

Second 

Blue  Win 

6 

7 

13 

Blue  Loss 

4 

3 

7 

Total 

10 

10 

20 

3.  By  Team 

Outcomes  varied  somewhat  by  team,  but  since  each  team  (Red  or  Blue)  played 
together  for  only  two  games,  it  is  difficult  to  draw  very  strong  conclusions  from  these 
results.  Indeed,  four  Blue  teams  won  both  their  games.  One  Blue  team  lost  both.  The 
five  remaining  teams  split  their  games  evenly.  One  player  (“a”)  served  on  three  of  the 
four  winning  Blue  teams,  but  also  played  with  the  team  that  lost  both  games  (but  no 
teams  with  split  records).  Two  players  (“b”  and  “e”)  served  on  two  winning  teams  and 
also  on  two  teams  with  split  records,  but  no  losing  teams. 

Teams  were  ranked  using  a  non-parametric  (Friedman)  test  for  multiple  related 
samples.  Rankings  varied  from  a  high  of  7.25  for  “ab,”  “ac,”  “ae,”  and  “be”  to  a  low  of 
2.25  for  “ad.”  All  other  teams  received  a  4.75  ranking  (the  highest  rank  is  best).  Despite 
the  range  of  these  ranks,  the  test  failed  to  find  significant  differences  among  the  10  teams. 

Table  9  presents  the  number  of  wins  by  each  Blue  team  (labeled  by  the  players  on 
the  team).  Of  course,  the  same  table  can  be  interpreted  as  a  description  of  the  Red 


2  Extreme  in  the  sense  that  one  group  loses  no  more  than  two  while  the  other  loses  at  least  five. 


15 


UNCLASSIFIED 


UNCLASSIFIED 


defensive  teams  as  well.  Table  10  shows  the  number  of  winning  games  recorded  by  each 
player.  Each  played  eight  games  as  Blue  and  twelve  games  as  Red. 


Table  9.  Blue  Team  Totals  (Wins  and  Losses) 


Outcome 

Team 

ab 

ac 

ad 

ae 

be 

bd 

be 

cd 

ce 

de 

Wins 

2 

2 

0 

2 

i* 

1* 

2 

i 

i* 

i* 

Losses 

0 

0 

2 

0 

i 

1 

0 

i 

i 

i 

*  The  single  win  occurred  under  enhanced  SA  (high  visibility) 


Table  10.  Player  Totals  (Blue  and  Red  Wins) 


Outcome 

Player 

a 

b 

c 

d 

e 

Blue  wins 

6 

6 

5 

3 

6 

Red  wins 

5 

5 

4 

2 

5 

Total 

11 

11 

9 

5 

11 
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IV.  ANALYSIS  OF  TACTICS 

A.  VISIBILITY  LEVEL 

After  all  the  exercises  were  completed,  we  reviewed  the  logger  files  in  an  attempt 
to  isolate  those  tactics  that  were  different  under  low  situational  awareness  exercises  from 
those  used  under  enhanced  or  high  SA  exercises.  Since  the  Red  forces  were  unaware  of 
the  visibility  level,  their  tactics  were  assumed  to  be  the  same  under  both  conditions. 

Review  of  the  tactics  used  by  the  Blue  force  revealed  an  interesting  and  obvious 
difference  between  high  SA  tactics  and  low  SA  tactics. 

Under  high  SA  conditions,  the  Blue  force  tended  to  attack  Red  forces  using 
coordinated  attacks.  In  few  cases  were  less  than  two  Blue  DI  deployed  in  an  initial  attack 
on  a  Red  objective.  Figure  3  shows  an  example  of  a  coordinated  attack  by  Blue  forces  on 
a  Red  position  in  a  high  SA  exercise.  Notice  that  Blue  forces  have  surrounded  the  Red 
stronghold  at  the  entrance  to  the  staircase.  In  this  particular  instance,  Blue  assaults  the 
Red  position  with  two  units  advancing  simultaneously  in  a  staggered  formation.  In  the 
attack,  Blue  was  successful  in  annihilating  the  Red  position  while  losing  one  unit. 

On  the  other  hand,  under  low  SA  conditions,  the  overwhelming  initial  tactic  that 
Blue  forces  used  was  to  send  certain  “lone-scouts”  to  discover  what  positions  were  being 
held  by  Red  forces.  After  discovering  Red  positions,  Blue's  tactics  proceeded  the  same 
as  under  high  SA  situations  (Blue  assumed  Red  positions  would  not  change,  so  they 
proceeded  as  if  they  knew  all  Red  positions  for  the  remainder  of  the  game).  Figure  4 
shows  an  example  of  the  use  of  the  “lone-scout”  tactic  in  a  low  SA  exercise.  Since  Blue 
does  not  know  the  location  of  Red  forces,  it  sends  units  to  discover  the  positions  being 
held  by  Red.  The  two  Blue  units  located  at  the  northern  windows  are  there  obviously  to 
see  if  the  respective  rooms  are  occupied.  Notice  that  each  room  is  being  monitored  by  a 
single  Blue  unit  compared  to  the  massed  convergence  of  Blue  units  as  seen  in  Figure  3. 
The  single  Blue  unit  labeled  “SCOUT”  in  Figure  4  is  shown  shot  on  approach  to  the 
building  by  a  Red  unit  looking  out  of  a  window  to  the  south.  The  line  that  traces  between 
the  Red  unit  to  the  south  and  the  Blue  scout  is  the  actual  projectile  path  as  shown  on  the 
DISAF  station  when  the  shot  was  fired.  Since  the  DISAF  station  shows  the  trajectory  of 
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the  shot,  it  becomes  a  simple  deduction  for  Blue  to  detennine  the  Red  position  once  a 
shot  was  fired.  In  contrast,  under  high  SA  condition,  Blue  rarely  gets  killed  on  approach 
to  the  building  since  the  approach  routes  will  be  chosen  to  reduce  his  vulnerability.  A 
few  seconds  after  this  particular  screen-capture  was  taken,  the  Blue  unit  to  the  northeast 
was  killed  by  the  Red  unit  hiding  inside  the  small  closet  when  the  Red  moved  up  to  get 
line-of-sight  to  that  window — another  scout  casualty. 


Figure  3.  Example  of  Coordinated  Assault 


18 


UNCLASSIFIED 


UNCLASSIFIED 


Figure  4.  Example  of  the  “Lone-Scout”  Tactic  Used  In  Low  Visibility  Exercises 

Given  the  above  observations,  we  speculated  the  following:  Blue  tactics  under 
low  SA  and  high  SA  are  the  same,  after  an  “initial  cost”  has  been  incurred  by  Blue  under 
low  SA  to  locate  Red  positions. 

In  an  attempt  to  quantify  our  speculation,  we  set  out  to  tabulate  the  total  number 
of  Blue  forces  that  were  lost  as  part  of  the  initial  cost  expended  by  Blue  forces  to  identify 
Red  positions.  If  indeed  the  initial  cost  was  the  only  difference  in  tactics  under  high  and 
low  S A  conditions,  the  overall  initial  cost  sum  should  be  close  to  1 8  (recall  from  Chapter 
III  that  Blue  forces  suffered  18  more  losses  under  low  visibility  situations  than  they  did 
under  high  SA  situations). 

To  tabulate  the  initial  cost  Blue  forces  suffered  under  low  SA  conditions,  we 
replayed  all  logger  files  for  low  SA  games  and  counted  all  Blue  forces  lost  to  Red 
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positions  that  were  unknown  to  Blue  at  the  time.  Red  positions  became  known  when  a 
Blue  unit  had  either  line-of-sight  to  the  Red  unit  or  when  the  Red  unit  fired  his  weapon. 
Red  units  could  also  occupy  multiple  unknown  positions  if  they  moved  into  a  new 
position  without  being  observed  by  Blue  forces.  Indeed,  a  single  Red  unit  scored 
multiple  Blue  scout  kills  in  several  trials  as  a  result  of  successful  movement. 

The  total  initial  cost  Blue  forces  suffered  during  the  low  SA  exercises  is  shown  in 
Table  11.  Total  Blue  casualties  resulting  from  unknown  Red  positions  for  the  10  low  SA 
games  was  found  to  be  18.  This  is  exactly  the  difference  in  Blue  casualties  between  the 
low  SA  and  high  SA  games.  This  fact  lends  credence  to  our  speculation  that  the  only 
difference  in  tactics  under  high  SA  and  low  SA  exercises  was  the  “scouting  out”  of  Red 
positions  by  Blue  scouts  in  low  SA  exercises.  Moreover,  14  Blue  scouts  were  killed 
outside  of  the  building  held  by  Red,  which  indicates  that  the  approach  was  particularly 
dangerous  in  the  low  SA  exercises. 


Table  11.  Blue  Casualties  Due  to  Unknown 
Red  Positions  in  Low  SA  Games 


Low  VIS  Trials 

Blue  Casualties  Due  to 
Unknown  Red  Positions 

Game  01 

2 

Game  02 

0 

Game  03 

2 

Game  04 

0 

Game  05 

6 

Game  06 

1 

Game  07 

1 

Game  08 

3 

Game  094 

2 

Game  10 

1 

Total 

18 

The  hypothesis  that  Blue  suffers  an  “initial  cost”  in  order  to  locate  Red  positions 
under  a  low  visibility  condition  may  be  tested  with  the  proposed  game  matrix  shown  in 
Table  12.  Blocks  1  and  2  represent  the  same  conditions  outlined  in  this  report.  Blocks  3 
and  4  introduce  two  additional  units  on  the  Blue  side.  Since  Blue  forces  suffered  18 
additional  losses  in  the  10  low  SA  games,  or  1.8  additional  losses  per  game,  the  two 
additional  units  proposed  for  Blue  in  blocks  3  and  4  are  meant  to  equalize  the  outcome  in 
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terms  of  the  number  of  Blue  wins.  Additionally,  by  comparing  block  1  to  block  3  and 
block  2  to  block  4,  we  can  identify  the  effect  of  an  increase  of  25  percent  in  Blue  forces. 


Table  12.  Proposed  Game  Matrix  To 
Test  the  “Lone-Scout”  Hypothesis 


Block 

Visibility  Level 

Blue  Units 

1 

High 

8 

2 

Low 

8 

3 

High 

10 

4 

Low 

10 

B.  EFFECT  OF  RED  POSITIONS 

From  our  experience  during  the  game  trials  and  the  subsequent  review  of  the 
logger  playbacks,  we  noticed  that  some  positions  in  the  building  were  occupied  more 
frequently  by  insurgency  forces.  Clearly,  these  positions  were  perceived  to  favor  Red. 
Several  factors  are  involved  in  these  choices,  including:  defensive  and  offensive 
capabilities,  possibility  of  cooperation  with  other  Red  players,  player  personality, 
previous  experience  at  the  same  location,  etc.  Therefore,  during  our  review  of  logger 
playbacks,  the  positions  held  by  Red  and  the  number  of  Blue  kills  achieved  by  each 
location  were  carefully  recorded  in  order  to  determine  whether  any  bias  existed  between 
the  high  visibility  and  low  SA  trials  as  a  result  of  the  positions  held. 

Figure  5  shows  the  layout  of  the  building  that  insurgency  forces  occupy.  The 
basic  layout  contains  four  large  rectangular  sections,  which  roughly  correspond  to 
“Cooperation  Zones,”  indicated  as  A  through  D.  The  individual  rooms  within  each  zone 
are  connected,  allowing  Red  players  within  the  same  zone  to  move  about  and  cover  each 
other.  The  individual  location  held  by  Red  is  identified  by  a  numerical  suffix  to  the  zone. 
Note  that  in  the  room  that  containing  position  B3,  located  within  the  third  large 
rectangular  section  from  the  top,  actually  belongs  to  zone  B  since  it  is  only  connected  to 
the  large,  open  room  to  the  north. 
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Figure  5.  Codes  Assigned  To  Various 
Positions  Within  the  Building 


The  location  dependent  results  are  tabulated  in  Tables  13  and  14.  “Frequency  of 
Occupation”  counts  the  instances  of  Red  units  holding  each  position.  “Blue  Casualties” 
counts  the  number  of  Blue  units  killed  from  each  position  occupied  by  Red.  The  casualty 
figure  for  the  scout  and  non-scout  kills  are  reported  separately  for  the  low  SA  exercises. 
In  addition  to  the  raw  data  on  locations  and  casualties,  two  measures  of  performance  are 
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listed  in  Table  13:  “Percentage  of  Games  Occupied”  and  “Efficiency  Ratio.”  First, 
“Percentage  of  Games  Occupied”  is  derived  by  taking  the  total  number  of  occupations  for 
each  location  for  both  high  visibility  and  low  SA  and  dividing  it  by  the  total  number  of 
trials.  It  is  a  measure  of  how  often  Red  holds  a  position  or  the  probability  that  a  Red  unit 
will  be  found  at  any  given  location  during  a  trial.  Second,  “Efficiency  Ratio”  is  derived 
by  taking  the  ratio  of  the  total  number  of  non-scout  Blue  casualties  to  the  total  number  of 
Red  occupations  for  each  position.  This  number  could  be  interpreted  as  the  expected 
number  of  Blue  kills  for  each  position  during  a  game. 

Detail  reporting  of  occupations  and  kill  records  for  each  individual  location  is 
shown  in  Table  13.  Comparing  the  frequency  of  occupation  for  positions  that  are 
occupied  25  percent  of  the  time  or  more  reveals  very  little  difference  between  high 
visibility  and  low  SA.  The  maximum  difference  in  the  number  of  occupations  for  these 
positions — A2,  B3,  C2,  C4,  C5,  and  C6 — is  only  1.  Therefore,  the  players  did  not 
inadvertently  favor  some  positions  over  others  from  high  visibility  to  low  SA.  It  can  also 
been  seen  that  four  out  of  six  of  these  highly  occupied  positions  had  efficiency  ratios  of 
greater  than  one,  while  for  the  less  occupied  positions,  only  B 1  had  an  efficiency  ratio  of 
greater  than  one. 

Table  14  shows  the  data  in  tenns  of  cooperation  zones  A  through  D.  Clearly, 
zone  C  is  the  most  occupied  of  the  four  zones.  The  frequency  of  occupation  and  non¬ 
scout  Blue  casualties  are  comparable  for  high  visibility  and  low  SA  in  zone  C.  Zone  C 
also  has  the  highest  efficiency  ratio  of  all  the  zones. 


Table  13.  Red  Frequency  of  Occupation  and  Blue  Casualties  by  Position 


Red 

Position 

Frequency  of 
Occupation 

Blue  Casualties 

Scout 

Percentage 
of  Games 
Occupied 

Efficiency 

Ratio 

HI-VIS 

LOW-VIS 

HI-VIS 

LOW-VIS 

LOW-VIS 

A1 

1 

0 

1 

5% 

0.00 

A2 

3 

4 

2 

5 

1 

35% 

0.00 

A3 

1 

3 

0 

1 

3 

20% 

0.25 

A4 

1 

3 

0 

3 

1 

20% 

0.75 

A5 

1 

0 

1 

5% 

0.00 

A6 

1 

0 

5% 

0.00 

B1 

3 

1 

6 

0 

0 

20% 

1.50 

B2 

1 

1 

0 

0 

1 

10% 

0.00 

B3 

3 

3 

1 

0 

2 

30% 

0.17 
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Table  13.  (continued) 


Red 

Position 

Frequency  of 
Occupation 

Blue  Casualties 

Scout 

Percentage 
of  Games 
Occupied 

Efficiency 

Ratio 

HI-VIS 

LOW-VIS 

HI-VIS 

LOW-VIS 

LOW-VIS 

Cl 

1 

1 

0 

2 

1 

10% 

1.00 

C2 

4 

3 

2 

4 

1 

35% 

0.86 

C3 

1 

1 

1 

0 

0 

10% 

0.50 

C4 

2 

3 

6 

2 

1 

25% 

1.60 

C5 

3 

2 

6 

8 

0 

25% 

2.80 

C6 

4 

4 

7 

11 

1 

40% 

2.25 

C7 

2 

1 

2 

0 

1 

15% 

0.67 

D1 

1 

1 

0 

1 

2 

10% 

0.50 

D2 

3 

1 

4 

0 

1 

20% 

1.00 

D3 

1 

0 

0 

5% 

0.00 

Table  14.  Red  Frequency  of  Occupation  and  Blue  Casualties  by  Zone 


Red 

Position 

Frequency  of 
Occupation 

Blue  Casualties 

Scout 

Efficiency 

Ratio 

HI-VIS 

LOW-VIS 

HI-VIS 

LOW-VIS 

LOW-VIS 

A 

6 

12 

2 

9 

7 

0.61 

B 

7 

5 

7 

0 

3 

0.58 

C 

17 

15 

24 

27 

5 

1.59 

D 

4 

3 

4 

1 

3 

0.71 

Total 

34 

35 

37 

37 

18 
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V.  CONCLUSIONS 


This  CRP  was  the  third  in  a  series  dealing  with  situational  awareness.  The  first 
two  were  based  on  small  armor  engagements.  They  were  conducted  using  ModSAF 
alone.  This  was  the  first  that  attempted  to  integrate  a  second  simulation,  the  Soldier 
Visualization  station,  into  a  distributed  environment  along  with  a  ModSAF  derivative. 
Also,  this  was  the  first  that  used  (exclusively)  dismounted  infantry  as  the  experimental 
entity.  The  conclusions  given  here  have  as  much  to  do  with  the  players  opinions  of  the 
two  models  as  useful  tools  for  MOUT  analyses  as  they  have  to  do  with  situational 
awareness.  Indeed,  the  exercise  was  largely  one  that  made  use  of  a  MOUT  scenario, 
hinged  on  situational  awareness,  to  evaluate  these  two  models. 

A.  MODELS 

These  experiments  were  conducted  with  opposing  forces  using  two  very  different 
simulators,  in  addition  to  having  different  force  sizes,  missions,  and  situational 
awareness.  Each  simulator  had  characteristics  and  limitations  that  affected  outcomes. 
For  example,  DISAF  entities  reacted  to  situations  much  more  rapidly  than  those  in  SVS. 
This  is  largely  due  to  the  automated  behavior  of  the  former — whereas  the  latter  depend 
on  direct  human  intervention.  Nonetheless,  some  form  of  “titration”  was  exercised  in  the 
pilot  tests  preceding  the  experiment  in  order  to  find  a  balance  between  the  two 
simulations.  This  involved  selecting  force  sizes  and  capabilities  that  produced  a 
relatively  “level  playing  field.” 

1.  DISAF 

DISAF  provides  greater  flexibility  for  an  operator  to  control  DI  units,  rather  than 
individual  soldiers.  For  example,  fire  teams  can  follow  prescribed  paths  in  a  coordinated 
manner  and  can  be  instructed  to  perform  relatively  sophisticated  functions,  such  as 
clearing  a  room.  Specific  behaviors  such  as  firing  or  otherwise  reacting  to  the  dynamic 
situation  are  programmed  into  the  simulation. 

The  tactical  situation  is  shown  on  PVD,  which  is  essentially  a  map  with  icons 
showing  soldier  locations.  Depending  on  the  situational  awareness  level  for  the 
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particular  run,  icons  would  show  all  soldiers,  or  only  friendly  soldiers  and  those  enemy 
units  in  direct  sight  of  a  friendly.  Control  was  exercised  using  familiar  computer 
interface  tools  such  as  selecting  from  a  menu  of  possible  commands  or  designating  points 
on  the  map  with  mouse  clicks. 

The  operator’s  ability  to  exercise  fine  control  and  react  to  changes  was  found  to 
be  very  limited.  In  previous  studies  of  tank  battles  using  ModSAF  a  “MOVE”  command 
(to  a  specific  point  on  the  map  display,  selected  by  a  mouse  click  as  opposed  to  a  combat 
instruction)  was  commonly  used  to  quickly  direct  tanks  and  override  any  previous 
commands.  This  command  did  not  work  for  individual  soldiers  (or,  at  least,  the  study 
team  was  unable  to  make  it  work).  In  these  experiments,  operators  entered  paths  and 
commanded  the  soldiers  to  follow  the  paths.  The  command  process  was  cumbersome 
and  commands  were  not  easily  changed  or  overridden.  This  lack  of  “micro”  control 
proved  awkward  in  MOUT  scenarios. 

DISAF  has  a  few  complex  behaviors,  but  these  were  not  found  to  be  useful.  For 
example,  the  “CLEAR  ROOM”  command  referred  to  above  was  not  used.  This  may 
have  had  more  to  do  with  the  time  and  patience  required  to  implement  the  command  than 
its  lack  of  utility.  More  generally,  operators  elected  not  to  issue  commands  to  the  entire 
fire  team  but  instead  elected  to  control  each  member  individually.  This  was  more 
cumbersome  and  time  consuming,  but  was  the  only  way  to  achieve  coordination  adapted 
to  the  situation.  Even  so,  coordination  was  difficult.  There  were  no  commands  that 
directed  soldiers  to  act  together,  e.g.,  to  move  or  to  attack  when  another  soldier  reached  a 
certain  position. 

Some  tactics  that  would  have  been  logical  to  employ  were  not  because  DISAF  did 
not  have  the  tools.  Perhaps  the  best  example  of  this  is  the  inability  to  use  protected  or 
semi-protected  positions.  Specifically,  it  was  not  possible  to  quickly  peer  around  comers 
or  to  find  a  partly  protected  location  from  which  to  fire. 

In  summary,  DISAF  does  not  seem  well  adapted  to  the  rapid,  precise,  and 
dynamic  movements  and  decisions  that  might  be  required  in  a  MOUT  environment.  For 
these  experiments,  the  DISAF  operators  constructed  planned  maneuvers,  which  worked 
reasonable  well  when  enemy  locations  were  known  and  the  enemy  soldiers  were  not 
moving. 
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2.  SVS 

The  Soldier  Visualization  Station  is  quite  different  from  DISAF:  one  operator 
controls  one  soldier;  the  controls  (a  joystick  with  a  trigger,  basically)  work  directly  on 
soldier  movement  and  actions  (rather  than  allowing  pre-planned  courses  of  action);  and 
the  situation  display  consists  of  the  soldier’s  view  (rather  than  a  map-like  display).  This 
allows  dynamic  and  reactive  control  that  is  not  possible  with  DISAF. 

There  are,  however,  significant  SVS  limitations.  Probably  the  greatest  is 
“awareness.”  The  single  computer  screen  does  show  much  peripheral  vision,  and  it  is  not 
possible  to  rotate  the  soldier’s  head  (separately  from  his  body)  to  look  around.  The 
controls  are  somewhat  “jumpy,”  so  that  controlling  fine  motion  and  aiming  at  small 
targets  can  be  difficult. 

In  addition,  there  are  simulation  implementation  problems.  Soldiers  become 
“stuck”  in  walls  upon  impact  and  may  even  protrude  through  the  other  side  of  a  wall 
(where  they  are  vulnerable).  During  these  exercises  SVS  would  freeze  occasionally  for  a 
few  seconds,  apparently  when  network  traffic  was  heavy. 

In  summary,  SVS  is  much  better  than  DISAF  in  responding  to  individual  control, 
but  still  has  limitations. 

B.  SITUATIONAL  AWARENESS 

The  limitations  of  the  simulators  and  the  use  of  a  single  limited  scenario  precludes 
drawing  general  conclusions  about  how  to  conduct  MOUT  operations  or  the  absolute 
value  of  awareness  of  enemy  positions.  Instead,  this  small  experiment  suggests  ideas  that 
might  be  analyzed  more  thoroughly,  including: 

•  Tactics  might  vary  depending  on  situational  awareness,  and  more  effective 
tactics  might  be  used  with  greater  situational  awareness. 

•  The  unprotected  approach  to  a  defended  building  is  highly  dangerous,  and 
might  benefit  greatly  from  awareness  of  enemy  locations  within  the  building. 
(It  should  be  noted  that  in  these  experiments  the  defending  force  did  not  have 
sufficient  manpower  to  cover  all  possible  approach  paths.) 

In  these  experiments,  the  Blue  force  used  much  better  coordinated  attacks  when 
the  Red  positions  were  known  than  when  they  were  not.  When  the  positions  were  not 
known,  Blue  actions  tended  to  be  exploratory  and  frequently  resulted  in  one  or  more 
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scout  soldiers  getting  killed.  When  positions  were  known,  Blue  seldom  used  only  one 
soldier  to  attack  an  enemy  position. 

The  difference  in  tactics  was  most  notable  in  approaching  the  building.  If 
positions  were  known.  Blue  would  generally  approach  the  building  from  an  unobserved 
direction  and  mount  a  coordinated  attack  after  reaching  the  building  perimeter.  If  enemy 
positions  were  not  known,  Blue  would  most  often  send  soldiers  individually  to  approach 
the  building  until  a  safe  approach  path  was  found. 

These  experiments  did  not  resolve  questions  involving  tactics  and  effectiveness 
within  the  building.  In  particular,  Blue  losses  in  approaching  the  building  without 
situational  awareness  might  have  occurred  within  the  building  (in  discovering  enemy 
positions),  even  if  the  force  was  able  to  approach  without  losses. 
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